[AMD] Update MiniMax-M3 FP8 MI355X ATOM image and serving args (0630) - #1968
Conversation
- Bump image rocm/atom-dev:MiniMax-M3-20260623 → MiniMax-M3-20260630 - Add OPT_ARGS: --online_quant_config (ptpc_fp8, excluding non-MoE layers) and --hf-overrides (use_index_cache, index_topk_freq=4) - Replace AITER_QUICK_REDUCE_CAST_BF16_TO_FP16=0 + ATOM_M3_SPARSE_USE_ASM_PA=1 with ATOM_FORCE_ATTN_TRITON=1 - Apply to both STP (atom.sh) and MTP/EAGLE3 (atom_mtp.sh) scripts Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook If it is not, please create a PR first before we can merge your single node PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you
PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. If re-running failed jobs is attempted, PR authors are responsible for ensuring it passes. See GitHub's docs on re-running failed jobs: https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow As a rule of thumb, generally, PR authors should request a review & get a PR approval from the respective companies' CODEOWNERS before requesting a review from core maintainers. If additional help is needed, PR authors can reach out to core maintainers over Slack. 感谢你的贡献！对于 vLLM 与 SGLang，请确保你的 recipe 与官方 vLLM recipes 和/或 SGLang cookbook 保持一致 如果不一致，请先创建一个 PR，之后我们才能将你的单节点 PR 合并到 master 分支。让我们确保文档保持一流水准，使整个 ML 社区都能从你的辛勤工作中受益！谢谢
PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。如果选择重新运行失败的任务，PR 作者有责任确保其最终通过。参见 GitHub 关于重新运行失败任务的文档：https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow 一般而言，PR 作者应先向相应公司的 CODEOWNERS 请求审阅并获得 PR 批准，然后再请求核心维护者审阅。 如需更多帮助，PR 作者可通过 Slack 联系核心维护者。 |
1 similar comment
|
Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook If it is not, please create a PR first before we can merge your single node PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you
PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. If re-running failed jobs is attempted, PR authors are responsible for ensuring it passes. See GitHub's docs on re-running failed jobs: https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow As a rule of thumb, generally, PR authors should request a review & get a PR approval from the respective companies' CODEOWNERS before requesting a review from core maintainers. If additional help is needed, PR authors can reach out to core maintainers over Slack. 感谢你的贡献！对于 vLLM 与 SGLang，请确保你的 recipe 与官方 vLLM recipes 和/或 SGLang cookbook 保持一致 如果不一致，请先创建一个 PR，之后我们才能将你的单节点 PR 合并到 master 分支。让我们确保文档保持一流水准，使整个 ML 社区都能从你的辛勤工作中受益！谢谢
PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。如果选择重新运行失败的任务，PR 作者有责任确保其最终通过。参见 GitHub 关于重新运行失败任务的文档：https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow 一般而言，PR 作者应先向相应公司的 CODEOWNERS 请求审阅并获得 PR 批准，然后再请求核心维护者审阅。 如需更多帮助，PR 作者可通过 Slack 联系核心维护者。 |
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| - "Bump image to rocm/atom-dev:MiniMax-M3-20260630 for both fp8 atom entries" | ||
| - "Add OPT_ARGS: pass --online_quant_config '{\"global_quant_config\": \"ptpc_fp8\", \"exclude_layer\": [\"lm_head\", \"model.embed_tokens\", \"vision_tower\", \"multi_modal_projector\", \"patch_merge_mlp\", \"*block_sparse_moe\"]}' and --hf-overrides '{\"use_index_cache\": true, \"index_topk_freq\": 4}' to both scripts" | ||
| - "Replace AITER_QUICK_REDUCE_CAST_BF16_TO_FP16=0 and ATOM_M3_SPARSE_USE_ASM_PA=1 with ATOM_FORCE_ATTN_TRITON=1" | ||
| pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/1968 |
There was a problem hiding this comment.
🔴 The new perf-changelog entry uses pull/PLACEHOLDER as its pr-link, but utils/validate_perf_changelog.py only accepts either the canonical pull/1968 or the literal placeholder XXX (PR_LINK_PLACEHOLDERS at lines 24-27). This will fail the changelog validation gate — replace PLACEHOLDER with XXX (auto-canonicalized to pull/1968 on merge) or with 1968 directly.
Extended reasoning...
What the bug is
perf-changelog.yaml:4362 sets:
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/PLACEHOLDERutils/validate_perf_changelog.py defines the accepted pre-merge placeholders at lines 24-27:
PR_LINK_PLACEHOLDERS = {
"XXX",
"https://github.com/SemiAnalysisAI/InferenceX/pull/XXX",
}validate_added_pr_link() (lines 144-160) requires the link to be either the canonical https://github.com/SemiAnalysisAI/InferenceX/pull/{pr_number} or a member of PR_LINK_PLACEHOLDERS. The value .../pull/PLACEHOLDER is neither, so the check raises ChangelogValidationError.
Where it triggers
utils/merge_with_reuse.shcallsprepare_perf_changelog_merge.py(bothresolve-conflictandcanonicalizesubcommands), which invokecompare_entries()→validate_added_pr_link().canonicalize_appended_links()also explicitly checksif link not in PR_LINK_PLACEHOLDERS. Sincemerge_with_reuse.shis the standard merge path, this blocks merging.- Convention across the file (see e.g. entries at lines 4337/4345/4353 with real PR numbers, and other in-flight entries with
XXX) confirms thatXXXis the accepted pre-merge placeholder — a real PR number is baked in on merge.
Step-by-step proof
- On merge,
merge_with_reuse.shrunsprepare_perf_changelog_merge.py canonicalize. - It reads the new entry with
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/PLACEHOLDER. validate_added_pr_link()computescanonical = 'https://github.com/SemiAnalysisAI/InferenceX/pull/1968'.- It checks
link == canonical→ False (PLACEHOLDER≠1968). - It checks
link in PR_LINK_PLACEHOLDERS = {'XXX', '.../pull/XXX'}→ False. - The check at line 156 raises
ChangelogValidationError: new PR entry must use 'https://github.com/SemiAnalysisAI/InferenceX/pull/1968' or an XXX placeholder; found 'https://github.com/SemiAnalysisAI/InferenceX/pull/PLACEHOLDER'.
How to fix
Replace PLACEHOLDER with the literal string XXX:
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXXThe canonicalize step will rewrite it to .../pull/1968 at merge time. Alternatively, bake in 1968 directly.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=28507173993 |
|
/reuse-sweep-run |
There was a problem hiding this comment.
[ x] Verified that as of the moment of typing this, this is the latest version of PR_REVIEW_CHECKLIST.md
[x ] Verified that the general code quality meets the InferenceX standard and does not make the code quality any worse.
[x ] Verified that this PR has passed PR validation: https://github.com/SemiAnalysisAI/InferenceX/actions/runs/28507173993
[x ] Verified that this PR passes evals: https://github.com/SemiAnalysisAI/InferenceX/actions/runs/28507173993
[ x] Verified that speculative decoding PRs uses chat templates to align the AL distribution to real world
[x ] If an company claims that they support vLLM/SGLang as first class LLM inference engines on their hardware, I have have verified that the respective vLLM/SGLang submission has been made before additional frameworks (TRT-LLM, ATOM, etc.). The only exceptions are for new hardware, such as MI455X UALoE72, Vera Rubin NVL72, Rubin NVL8, etc., and for new model architectures where there is an actual reason why vLLM/SGLang does not fundamentally support them yet.
Signed: @chunfangamd
3 participants
Summary
rocm/atom-dev:MiniMax-M3-20260623→MiniMax-M3-20260630for bothminimaxm3-fp8-mi355x-atomandminimaxm3-fp8-mi355x-atom-mtpOPT_ARGSto both scripts:--online_quant_config(ptpc_fp8, excluding non-MoE layers) and--hf-overrides(use_index_cache: true,index_topk_freq: 4)AITER_QUICK_REDUCE_CAST_BF16_TO_FP16=0+ATOM_M3_SPARSE_USE_ASM_PA=1withATOM_FORCE_ATTN_TRITON=1As a PR reviewer and CODEOWNER, I have reviewed this and have:
🤖 Generated with Claude Code